Rényi entropy and quantization for densities
نویسندگان
چکیده
Abstract—A random variable Z taking value in a finite, nonatomic measure space (X,M, μ) and whose distribution is absolutely continuous with respect to μ is to be described using N labels. We seek the labeling that minimizes the ρ-th moment of the μ-volume of the set of points in X that have the same label as Z. The large-N asymptotics of this minimum are expressed in terms of the Rényi entropy of order 1/(1 + ρ).
منابع مشابه
A Preferred Definition of Conditional Rényi Entropy
The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced a...
متن کاملEntropy Density and Mismatch in High-Rate Scalar Quantization with Renyi Entropy Constraint
Properties of scalar quantization with rth power distortion and constrained Rényi entropy of order α ∈ (0, 1) are investigated. For an asymptotically (high-rate) optimal sequence of quantizers, the contribution to the Rényi entropy due to source values in a fixed interval is identified in terms of the “entropy density” of the quantizer sequence. This extends results related to the well-known po...
متن کاملThe Rate of Rényi Entropy for Irreducible Markov Chains
In this paper, we obtain the Rényi entropy rate for irreducible-aperiodic Markov chains with countable state space, using the theory of countable nonnegative matrices. We also obtain the bound for the rate of Rényi entropy of an irreducible Markov chain. Finally, we show that the bound for the Rényi entropy rate is the Shannon entropy rate.
متن کاملHigh-resolution scalar quantization with Rényi entropy constraint
We consider optimal scalar quantization with rth power distortion and constrained Rényi entropy of order α. For sources with absolutely continuous distributions the high rate asymptotics of the quantizer distortion has long been known for α = 0 (fixed-rate quantization) and α = 1 (entropyconstrained quantization). These results have recently been extended to quantization with Rényi entropy cons...
متن کاملOptimal quantization for the one-dimensional uniform distribution with Rényi-α-entropy constraints
We establish the optimal quantization problem for probabilities under constrained Rényi-α-entropy of the quantizers. We determine the optimal quantizers and the optimal quan-tization error of one-dimensional uniform distributions including the known special cases α = 0 (restricted codebook size) and α = 1 (restricted Shannon entropy).
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2014